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Method and Apparatus for coding/decoding items of subtitling 
data 

The invention relates to a method and to an apparatus for 
coding/decoding items of subtitling data, in particular sub- 
titling and graphics for Blu-ray disc optical storage and 
recording . 



Background 

in the area of subtitling, for pre-recorded Audio-Visual (AV) 
material, conflicting requirements exist: On one hand, sub- 
titling data should be efficiently encoded, especially if a 
whole bouquet of subtitling services is to be provided for 
any given AV material, m this case, at least on average, 
very few bits are available per subtitling character. 
On the other hand, professional content owners want to have 
full control over the appearance of subtitling characters on 
screen, additionally they want to have at their command a 
rich set of special display effects from simple fading all 
through to genuine animations. Such high degree of design 
freedom and command normally is feasible only with high or 
very high subtitling bandwidth. 

Two main approaches exist in today's state of the art for 
subtitling pre-recorded AV data signals with separate subti- 
tling information: Subtitling can. be based on either pixel 
data or on character data, m both oases, subtitling schemes 
oomprise a general framework, which for instance deals with 
the synchronisation of subtitling elements along the AV time 
axis . 

In the character-based subtitling approach, e.g. in the 
TELETEXT system (see ETSI: ETS 300 706 Enhanced Teletext 
specification, May 1997) for European analog or digital TV 
strings are described by sequences of letter codes, e.g. 
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ASCII (see ISO/IEC 885$; American Standard Code for Informa- 
tion Interchange - ASCII) or UNICODE (see ISO/IEC 10646: In- 
formation technology Universal Multiple -Octet Coded Char- 
acter Set (UCS)) , which intrinsically allows for a very ef- 
s ficient encoding. But from character strings alone, subti- 
tling can not be converted into a graphical representation ! 
to be overlaid over video. For this, the intended character 
set/ font and some font parameters, most notably the font 
size, must either be coded explicitly within the subtitling 
10 bitstream or an implicit assumption must be made about them 
within a suitably defined subtitling context. Also, any sub- 
titling in this approach is confined to what can be ex- 
pressed with the letters and symbols of the specific font or 
fonts in use, 

15 The DVB Subtitling Specification (see ETSI: ETS 300 743 
Digital Video Broadcasting (DVB); Subtitling systems, Sep 
1397, and EP-A-0 745 307: Van der Meer et al, Subtitling 
transmission system), with its object types of * basic ob- 
ject, character' or 'composite object, string of character', 

20 constitutes another state-of-the-art example of character- 
based subtitling. 



In the pixel -based subtitling approach, subtitling frames 
are conveyed directly in the form of graphical representa- 

25 tions by describing them as (typically rectangular) regions 
of pixel values on the AV screen. Whenever and wherever any- 
thing is meant to be visible' in the subtitling plane super- 
imposed onto video, its pixel values must be encoded and 
provided in the subtitling bitstream, together with appro- 

30 priate synchronisation info. Obviously removing any limita- 
tions inherent with 3rd party defined fonts, the pixel-based 
approach carries the penalty of a considerably increased 
bandwidth for the proper subtitling data. Examples of pixel - 
based subtitling schemes can be found in DVD's * Sub-picture' 

35 concept (see DVD Forum: DVD Specifications for Read-Only 
Disc / Part 3 Video Specifications / Version 1.0 August 
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25>se) as well as in the 'bitmap object' concept of dvb sub- 
titling (see STS 300 743 and BP-A-0 745 307 mentioned 
above) . 



Invention 



A problem to be solved by the invention is to combine the 
efficient encoding of character -based subtitling with full 
control over the appearance of subtitling characters as is 
feasible with pixel-based subtitling, without significantly 
increasing the data amount required for transferring the 
necessary information. This problem is solved by the methods 
disclosed in claims 1 and 7. An apparatus that utilises the 
method of claim 1 is disclosed in claims. 

The invention is based on a pixel -based subtitling scheme. 
This subtitling system includes several components which al- 
low to include font support into an otherwise pixel -based 
subtitling scheme. This font support includes: 
a.l) A structure for Pont Describing Data for efficiently 
describing a set of font characters in pixel data form; 
a. 2) A structure for Pont Identification Data to uniquely 
identify a predefined font to be used; 

a. 3) A concept of having a font memory as a part of the 
overall memory area, wherein that font memory is dedicated 
to hold the font characters, and is not directly visible in 
the AV output ; 

a. 4) A structure for Character Referencing Data for «f£±. 
ciently referencing individual font characters from amongst 
the font or fonts stored in the font memory. 

Pont Describing Data as well as Character Referencing Data 
are transmitted or stored alongside AV data, whereby that 
transmission or storage has either the format of a nearly 
inseparable mix or uses completely separate transmission 
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channels or storage locations, or is a mix of both- 
At decoder side the Pont Describing Data cause a set of ar- 
bitrary character glyphs (graphical representation of a 
character) or other graphics building blocks to be loaded 
5 into the font memory. The number and design of character 
glyphs to be used in each individual case is completely un- 
der the control of the content provider. 

According to the invention, the Pont Describing Data consist 

10 of one or more character parameter parts each coiqprising 
character parameter sets of one ore more characters in the 
font and one or more character pixel data parts each com- 
prising the pixel data of one or more characters in the 
font. The pixel data of a character are represented as a 

15 character array, i.e. as a rectangular array of pixel val- 
ues, the array having a width and a height specific to the 
character, Each one of said character parameter sets in- 
cludes any combination of: 
c.l) The width of the character array; 

20 c.2) The height of the character array; 

c.3) The start address of the pixel data of the character 
relative to the character pixel data part containing it; 
c.4) A horizontal offset between the boundaries of the array 
and a character reference point; 

25 c.5) A vertical offset between the boundaries and the char- 
acter reference point; 

c.6) A horizontal increment describing the horizontal dis- 
tance between the character and those characters to either 
precede or succeed it. 

30 

The inventive use of a font memory provides an efficient re- 
alisation of pixel -based subtitle lettering, because the 
glyphs need only be transmitted once and thereafter are ref- 
erenced by relatively compact character references during 
35 the AV event. 

On the other hand, because glyphs are effectively provided 
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in pixel -based form, the appearance of subtitling is en- 
tirely put under content provider's control, and all prob- 
lems of font identification, font selection, font parametri- 
sat ion and character rendering, which normally come with, 
character-based schemes, are avoided advantageously. 

In this way, the invention actually combines the advantages 
of pure pixel-based and pure -character-based subtitling 
schemes, while mostly avoiding their respective shortcom- 
ings . 

in principle, the inventive method is suited for decoding 
items of subtitling data, including the steps: 

- retrieving items of Character Referencing Data that are- 
related to corresponding parts of a video or audio-visual ■'• 
data signal, which data items describe sequences of. charac- 
ters as well as information about where in pictures of said 
data signal and/or when and/or how to make the referenced- 
characters visible using a display memory; 

- deriving from said items of Character Referencing Data ■ 
items of Character Selecting information and Character Posi- 
tioning Information; 

- reading pixel data of said referenced characters as des- 
ignated by said items of Character Selection Information 
from a font memory; 

- writing said pixel data into said display memory as des- 
ignated by said items of Character Positioning Information. 

In principle the inventive apparatus is suited for decoding 
items of subtitling data, said apparatus including: 

- means for retrieving items of Character Referencing Data 
that are related to corresponding parts of a video or audio- 
visual data signal, which data items describe sequences of 
characters as well as information about where in pictures of 
said data signal and/or when and/or how to make the refer- 
enced characters visible using a display memory; 




means for: 

deriving from said items of Character Referencing Data 
items of Character Selecting Information and Character Posi 
tioning Information; 

reading pixel data of said referenced characters as des- 
ignated by said items of Character Selection Information 
from a font memory/ 

writing said pixel data into said display memory as des- 
ignated by said items of Character Positioning Information, 

Advantageous additional embodiments of the invention are 
disclosed in the respective dependent claims. 

Drawings 

Exemplary embodiments of the invention are described with 
reference to the accompanying drawings, which show ins 
Pig, 1 Inventive data structure; 

Fig. 2 Block diagram of the inventive subtitling system; 
Fig. 3 Example data structure for embedding a *font_id' 
into a DVD- ST % object_data_segment' . 

Exemplary embodiments 

As illustrated in Pig. 1, the Pont Describing Data 102 as 
well as the Character Referencing Data 103 are transferred, 
stored or recorded together with related AV data 101, 
whereby the transmission or storage can be anything between 
a nearly inseparable mix and the use of completely separate 
transmission channels or storage locations. 

At decoder side, as shown in Fig. 2, a subtitling stream 201 
passes through data separation means 202, which in turn pro- 
vides Character Referencing Data 203 and Font Describing 
Data 204. By passing a font describing data processing means 




205, the Pont Describing Data 204 cause a set of arbitrary 
character glyphs or other graphics building blocks to be 
loaded into a font memory 208. 

Advantageously, the number and design of character glyphs to 
be used in each individual use case is completely under con- 
tent provider's control. 

Optionally, to a font thus described and loaded into font 
memory 208, the above-mentioned Pont Identification Data can 
be associated. 

The Character Referencing Data 203 cause character referenc- 
ing data processing means .206 to copy individual subsets of 
the set of character glyphs denoted Character Describing 
Data 209 from font memory 208 into a display memory 207, 
which can be a part of the overall system memory. The con- 
tent of display memory 207 gets overlaid onto video and 
hence becomes a visible subtitle. 

Optionally, the character Referencing Data can contain ref- 
erences to the Pont Identification Data, thus allowing a 
subtitling decoder to decide whether a font required for 
rendering a specific subtitling stream must still be loaded • 
into font memory 208, or is already available for immediate 
use. 

Possible uses and modes of operation of the proposed subti- 
tlxng system can include, but are not limited to, one of- 
b.l) Pre-loading at least one font for use throughout a lona 
AV program; 

b.2) use of fonts containing more than one variant for at 
least one of the letters, the use of which includes, but is 
not limited to, subpixel -accurate letter positioning or em- 
phasis (bold/italic) support; 

b.3> Loading font subsets for parts of AV material (e.g 
movie chapters) in cases where sparse subsets of big fonts 
are used, like e.g. Asian fonts. 
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For the further structure of the Font Describing Data, sev- 
eral variants of specific embodiment are proposed as fol- 
lows . 

In a first variant, if the font is a proportional font where 
individual characters have variable width, all the character 
arrays are horizontally padded to be nominally of equal 
width, and the resulting padded character arrays are verti- 
cally concatenated into a font array. The font array is then 
line-scanned in conventional way to form a single character 
pixel data part. 

In another variant, all character arrays are vertically pad- 
ded to be nominally of equal height, and the resulting pad- 
ded character arrays are horizontally concatenated into a 
font array. The font array is then line-scanned in conven- 
tional way into a single character pixel data part. 
For both above variants, the single character pixel data 
part is preceded by a single . character parameter part com- 
prising the character parameter sets of all characters in 
the font. 

In another variant, the Font Describing Data are generated 
by alternately concatenating the character parameter sets 
and the character arrays, for all characters in the font. 

In another variant, the Font Describing Data are generated 
by first concatenating all the character parameter sets into 
a single character parameter part, and appending to that 
part a single character pixel data part comprising all the 
character arrays. 

In another variant, which may optionally extend all above 
variants, a UNICODE (see ISO/IEC 10646; Information technol- 
ogy Universal Multiple-Octet Coded Character Set (UCS)) 
code is associated to some or all of the characters of the 
font, and the UNICODE code is inserted and included at an 
identifiable position within that part of the Font Describ- 
ing Data which is associated with the character in question. 
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In another variant, which may optionally extend all above 
variants, a non-repetitive character identifier is associ- 
ated to every character of the font, and the identifier is 
inserted and included at an identifiable position within 
that part of the Font Describing Data which is associated 
with the character in question. 

In all above variants, the Pont Describing Data can either 
be 

d.l) directly transmitted using one codeword per data item, 
or they can be 

d.2) compressed by runlength coding, or they can be : 
d.3.) compressed by other methods for lossless compression 
suoh as the *zlib' method used in PNG (see W3C recommenda- 
tion, pm (Portable Network Graphics) Specification, Version 
1.0, 1996, http: //www. w3 . orcf/TR/REC-png . Pdf ) . 

For the structure of the Font Identification Data, several 
variants of specific embodiment are proposed as follows. 
In a first variant, the Font Identification Data structure 
is embodied as a l font_id' as defined in the » Portable Font 
Resource' (PFR) system (see Bit stream Inc. : TrueDoc PFR 
Specification, http://www.bitstream.aom/pfrspec/index.html) . 

In another variant, the Font Identification Data structure 
in the form of a PFR »font_id' is embodied into the above- 
mentioned DVB subtitling system, using a data structure as 
illustrated in Fig. 3. 

In another variant, the Font Identification Data structure 
is embodied as a "Universally Unique Identifier" as defined 
in (UUID in: IS0/IEC 11578 1 1996 , Information technology - 
Open Systems Interconnection - Remote Procedure Call (rpc) ) . 

In the context of the invention, the Character Referencing 
Data consist of a sequence of one or more character refer- 
ence groups each accompanied by group positioning, data, and 



• • 

each character reference group consists of. a sequence of on< 
or more character references each accompanied by character 
positioning data. 

5 The group positioning data can preferably be embodied as on< 
of: 

e.l) Absolute horizontal and vertical coordinates of a grouj 
reference point relative to the origin of the video image; 
e.2) Relative horizontal and vertical coordinates of the 
10 group reference point relative to the group reference point 
of the previous character reference group; 

e. 3) Relative horizontal and vertical coordinates relative 
to any other prescribed reference point. 

15 The character references can preferably be embodied as one 
Of: 

f . 1) Character indexes referring to the implicit position of 
the designated character within the Pont Describing Data; 

f . 2) Any kind of unambiguous character identifiers; 

20 f .3) ASCII codes if they have been unambiguously assigned to 
the characters; 

f .4) UNICODE codes if they have been unambiguously assigned 
to the characters. 

25 The character positioning data can preferably be embodied as 
one of: 

g. l) An automatic advance needing no additional individual 
character positioning data, the advance being deductible 
from the position of the character reference point of the 

30 previous character and from the horizontal increment of the 
character in question; 

g.2) An automatic advance with character position offset 
data, where for the horizontal as well as for the vertical 
position of the character a first value deduced from the po- 
35 sition of the character reference point of the previous 

character and from the horizontal increment of the character 




in question is added with a second value which is individur 
ally described in the character positioning data; 
g.3) Relative character positioning data applied relative to 
the character reference point of the previous character; 
g.4) Absolute character positioning data applied relative to 
the video image origin. 
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Claims 

1* Method for decoding items of subtitling data, character- 
ised by the steps: 
5 - retrieving (202) items of Character Referencing Data 

(103/ 203) that are related to corresponding parts of a 
video or audio-visual data signal (101) , which data items] 
(103/ 203) describe sequences of characters as well as 
information about where in pictures of said data signal 
10 and/or when and/or how to make the referenced characters 

. visible using -a display memory (207); 
. - deriving (20S) from. said items of Character Referencing 
Data (103/ 203) items of Character Selecting Information 
and Character Positioning Information; 
is .-** reading (206) pixel, data of said referenced characters as 
designated by said items of Character Selection Informa- 
tion from a font memory (208); 

- writing (20S) said pixel data into said display memory 

(207) as designated by said items of Character Position- 
al) ing Information. 

2> Method according to claim 1, wherein the following steps 
are carried out before retrieving (202) said items of 
Character Referencing Data (103/ 203) s 
25 - retrieving (202) items of Font Describing Data (102 , 204) 
related to corresponding ones of said items of Character 
Referencing Data (103/ 203); 

- writing (205) said items of Font Describing Data into 
said font memory (208) - 

30 

3- Method according to claim 1 or 2, wherein, after retriev- 
ing said items of Character Referencing Data (103, 203), 
the following steps are carried out: 

- checking whether or not said pixel data of said refer- 
35 enced characters are already stored in said font memory 

(208) ; 
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- if not true, retrieving (202) suah items. of Font Describ- 
ing Data (102, 204) which contain said referenced charac- 
ters; 

- writing said items of Pont Describing Data into said font 
memory (208) . 

4. Apparatus for decoding items of subtitling data, said ap- 
paratus including: 

- means (202) for retrieving items of Character Referencing 
Data (103, 203) that are related to corresponding parts 
of a video or audio-visual data signal (101), which data 
items (103, 203) describe sequences of characters as well 
as information about where in pictures of said data sig- . 
nal and/or when and/or how to make the referenced charac- 
ters visible using a display memory (207); 

- means (206) for: 

deriving from said items of Character Referencing Data 
(103, 203) items of Character Selecting Information and 
Character Positioning Information; 

reading pixel data of said referenced characters, as des- 
ignated by said items of character Selection Information . 
from a font memory (208) ; 

writing said pixel data into said display memory (207) as 
designated by said items of Character Positioning Infor- 
mation. 

5. Apparatus according to claim 4, wherein said means (202) 
for retrieving, before retrieving said items of Character 
Referencing Data (103, 203), retrieve items of Pont De- 
scribing Data (102, 204) related to corresponding ones of 
said items of Character Referencing Data (103, 203), said 
apparatus further including: 

means (205) for writing said items of Font Describing 
Data into said font memory (208) . 



Apparatua according to claim 4 or 5, further including 
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means for checking , after retrieving said items of Char- 
acter Referencing Data (103, 203) , whether or not said 
pixel data of said referenced characters are already 
stored in said font memory (208) , wherein, if not true, 
such items of Pont Describing Data (102, 204) are re- 
trieved that contain said referenced characters, and are 
written into said font memory (208) . 

7* Method for encoding subtitling data, characterised by the 
step: 

- attaching to a video or audio-visual data signal (101) 
related subtitling data including items of Character Ref- 
erencing Data (103, 203) and items of Font Describing 
Data (102, 204) , 

whereby said items of Character Referencing Data (103, 
203) describe sequences of characters as well as informa- 
tion about where in pictures of said data signal and/or 
when and/or how to make the referenced characters visible 
using, a display memory, said items of Character Referenc- 
ing Data including items of Character Selecting Informa- 
tion and Character Positioning Information, wherein said 
items of Character Selection Information can be used in a 
subtitle decoder for reading pixel data of said refer- 
enced characters from a font memory and said items of 
Character Positioning Information can be used in said 
subtitle decoder for writing said pixel data into said 
display memory, 

and whereby said items of Font Describing Data (102, 204) 
can be written in said subtitle decoder into said font 
memory for checking whether or not said pixel data of 
said referenced characters are already stored in said 
font memory and, if not true, retrieving such items of 
Font Describing Data (102, 204) which contain said refer- 
enced characters and writing said items of Font Describ- 
ing Data into said font memory • 
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8. A data carrier containing a video or audio-visual data 
signal (101) and related subtitling data that are encoded 
using a method according to claim 7. 




Abstract 

Subtitling can be based on either pixel data or on character 
data. Character data allow very efficient encoding, but frotr 
5 character strings alone, subtitling can not be converted 
into a graphical representation to be overlaid over video. 
The intended character set, font and e.g. font size, must 
either be coded explicitly within the subtitling bitstream 
or an implicit assumption must be made about them, m pixel- 

10 based subtitling, subtitling frames are conveyed directly in 
the form of graphical representations by describing them as 
(typically rectangular) regions of pixel values on the AV 
screen, at the cost of considerably increased bandwidth for 
the subtitling data. According to the invention, a font mem- ! 

15 ory is used that allows an efficient realisation of pixel- ! 
based subtitle lettering, because the glyphs need only be 
transmitted once and thereafter are referenced by relatively 
compact character references during the AV event. Thereby 
the invention combines the advantages of pure pixel -based 

20 and pure -character-based subtitling schemes, while mostly 
avoiding their respective shortcomings. 
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